Papers with context representation
Knowledgeable Reader: Enhancing Cloze-Style Reading Comprehension with External Commonsense Knowledge (P18-1)
Copied to clipboard
| Challenge: | a new model for reading comprehension integrates external commonsense knowledge . cloze-style reading comprehension is a language understanding task similar to question answering . |
| Approach: | They propose a reading comprehension model that integrates external commonsense knowledge in a cloze-style setting. |
| Outcome: | The proposed model improves results over a very strong baseline on a hard Common Nouns dataset, making it a strong competitor of more complex models. |
Exploiting Contextual Information via Dynamic Memory Network for Event Detection (D18-1)
Copied to clipboard
| Challenge: | Existing methods for event detection only process context once . a multi-hop mechanism to capture contextual information improves performance . |
| Approach: | They propose to use dynamic memory network to capture contextual information . they propose to model event triggers by identifying word or phrase which most represents it . |
| Outcome: | The proposed model achieves best F1 score compared to the state-of-the-art models. |
Leveraging Gloss Knowledge in Neural Word Sense Disambiguation by Hierarchical Co-Attention (D18-1)
Copied to clipboard
| Challenge: | Existing models for Word Sense Disambiguation use labeled data, but lack gloss knowledge. |
| Approach: | They propose a co-attention mechanism to generate co-dependent representations for context and gloss . they propose to incorporate gloss knowledge into neural networks for Word Sense Disambiguation . |
| Outcome: | The proposed model achieves state-of-the-art results on standard English all-words WSD datasets. |
Efficient Second-Order TreeCRF for Neural Dependency Parsing (2020.acl-main)
Copied to clipboard
| Challenge: | In the deep learning (DL) era, dependency parsing models are extremely simplified with little hurt on performance thanks to the remarkable capability of multi-layer BiLSTMs in context representation. |
| Approach: | They propose to extend the biaffine parser to a second-order TreeCRF extension to reduce the complexity of the inside-outside algorithm. |
| Outcome: | The proposed extension can be used to batchify the inside and Viterbi algorithms and avoid the complex outside algorithm via efficient back-propagation. |
Layer-Wise Multi-View Learning for Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to neural machine translation are limited to the topmost encoder layer’s context representation and cannot perceive the lower encoder layers. |
| Approach: | They propose a layer-wise multi-view learning approach to solve this problem by incorporating an auxiliary view into the model. |
| Outcome: | The proposed model can achieve stable results over multiple strong baselines and is agnostic to network architectures. |
Learning Decoupled Retrieval Representation for Nearest Neighbour Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to integrate external corpus are sparse in practical applications, and noises in low similarity retrieval could lead to severe performance degradation. |
| Approach: | They propose a method to integrate external corpus into k-nearest neighbor machine translation (kNNMT) instead of storing discrete word sequence, kNN-MT uses a pre-trained NMT model to force decoding the external corpi. |
| Outcome: | The proposed approach improves retrieval accuracy and BLEU score on five domains compared to vanilla kNNMT. |
Enhancing Neural Machine Translation Through Target Language Data: A kNN-LM Approach for Domain Adaptation (2025.acl-long)
Copied to clipboard
Abudurexiti Reheman, Hongyu Liu, Junhao Ruan, Abudukeyumu Abudula, Yingfeng Luo, Tong Xiao, JingBo Zhu
| Challenge: | Neural machine translation (NMT) has made significant progress in recent years, yet often suffers from translating in new domains, which is called domain adaptation. |
| Approach: | They propose a method that leverages semantically similar target language sentences in the kNN framework and generates a probability distribution over these sentences during decoding. |
| Outcome: | The proposed method generates a probability distribution over similar target language sentences and then interpolates with the model’s distribution. |
Counterfactual Generator: A Weakly-Supervised Method for Named Entity Recognition (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using labeled data, named entity recognition is labor-intensive, time-consuming and expensive. |
| Approach: | They propose a method which decomposes named entity into two parts: entity and context. |
| Outcome: | The proposed method improves the generalization ability of models under limited observational examples. |
Exploiting Sentential Context for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to exploit sentential context for machine translation are not well studied. |
| Approach: | They propose a shallow sentential context that exploits top encoder layer, and a deep sentential one that aggregates sentential representations from all internal layers. |
| Outcome: | The proposed model outperforms the strong Transformer model on the English-German and English-French benchmarks. |
Causal Intervention for Mitigating Name Bias in Machine Reading Comprehension (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing MRC models may overuse name information to make predictions, causing name bias . |
| Approach: | They propose a Causal Interventional paradigm for MRC to mitigate name bias by analyzing pre-trained knowledge and context representations. |
| Outcome: | The proposed model is robust to names and performs competitively on the original SQuAD. |